Logistics  Management  Institute 


A  Database  for  Decision- 
Making  in  Training  and 
Distributed  Learning 
Technology 


ro 


PR702T1 


April  1998 

MBTMBDTION  gTATEMEHT  A  j 

Appctnred  for  pr&bite  rolocnof  Virffillifl  StOUffcr 

DlsBtdfeiiecio  UnlSmttod  ® 


^  QUALITY  MSfEOIED  a 

LMI 


A  Database  for  Decision- 
Making  in  Training  and 
Distributed  Learning 
Technology 


PR702T1 
April  1998 


Virginia  Stouffer 


Prepared  pursuant  to  Department  of  Defense  Contract  DASW01-95-C-0019.  The  views  expressed 
here  are  those  of  the  Logistics  Management  Institute  at  the  time  of  issue  but  not  necessarily  those 
of  the  Department  of  Defense.  Permission  to  quote  or  reproduce  any  part  except  for  government 

purposes  must  be  obtained  from  the  Logistics  Management  Institute. 

Logistics  Management  Institute 
2000  Corporate  Ridge 
McLean,  Virginia  22102-7805 


Logistics  Management  Institute 


A  Database  for  Decision-Making  in  Training  and 
Distributed  Learning  Technology 

PR702T1/APRIL 1998 


Executive  Summary 


The  Under  Secretary  of  Defense  for  Personnel  and  Readiness  (USD[P&R])  has 
oversight  responsibility  for  the  military’s  huge  institutional  training  base  and  is 
responsible  for  advising  the  Secretary  of  Defense  on  resource-  and  readiness- 
related  issues.  With  the  annual  training  budget  running  at  $14  billion,  the  office  of 
the  USD(P&R)  is  concerned  with  ways  to  achieve  training  at  lower  cost  while 
maintaining  or  improving  effectiveness.  Technological  improvements  of  the 
1990s,  such  as  the  Internet,  offer  great  potential  to  reduce  the  costs  and  improve 
the  quaUty  of  training.  The  Under  Secretary  has  focused  sharply  on  encouraging 
the  military  services  to  exploit  this  technology  in  training. 

To  support  the  transition  to  increased  technology,  LMI  constructed  a  database  en¬ 
compassing  the  services’  and  DoD’s  institutional  skill  training  courses — over 
30,000  courses.^  Among  other  things,  this  database  is  useful  in  identifying  institu¬ 
tional  training  courses  for  distributed  learning  conversion  or  other  possible  cost¬ 
saving  changes.  Initial  questions  we  posed  to  the  database  included  the  following: 
Which  institutional  courses  offered  by  the  Marine  Corps,  Air  Force,  Navy,  and 
DoD  are  similar  to  distributed  learning  courses  planned  by  the  Army?  Can  we 
group  courses  together  to  create  consolidated  “high-load”  courses,  and  to  what 
extent  can  we  say  it  would  make  sense  to  use  technology  in  high-load  courses? 
Which  courses  use  a  form  of  “distributed  learning”  already?  Which  courses  are 
the  best  candidates  for  distributed  learning?  The  information  needed  to  address 
these  questions  was  incorporated  into  the  database  and  many  such  questions  were 
answered. 

As  P&R’s  distributed  learning  initiatives  continue  to  develop,  the  database  can  be 
useful  for  answering  staffing  questions  and  planning  transitions  to  technology- 
assisted  courses.  The  database  already  can  provide  some  information  to  help  de¬ 
cide  where  priorities  lie.  The  database  can  describe  which  course  subjects  are 
most  frequently  offered  in  schoolhouse  training;  how  geographically  dispersed  the 
training  is;  and  whether  enlisted  personnel,  officers,  or  other  people  comprise  the 


'  Institutional  training  refers  to  the  traditional  classroom  lecture  instruction  method,  delivered 
in  a  dedicated  school  facility,  rather  than  on  the  job  site. 


Ill 


majority  of  students.  The  database  can  reveal  which  courses  are  most  often  taken 
by  pilots,  by  medical  students,  and  by  other  career  choices,  and  how  long  the 
coursework  takes.  In  the  current  state  of  high  utilization  of  certain  military  spe¬ 
cialties  such  as  pilots,  this  database  could  begin  to  identify  and  quantify  ways  to 
reduce  hours  associated  with  training  and  increase  the  availability  of  pilots  for 
duty.  As  budget  planners  examine  base  realignment  and  closings,  this  database 
can  help  illustrate  where  alternative  schoolhouse  training  is  available  and  where  it 
could  be  lacking. 

Policy  and  budget  questions  that  are  likely  to  arise  could  be  addressed,  at  least 
partly,  with  the  assistance  of  the  database.  In  transitioning  to  a  distributed  learning 
curriculum,  planning  questions  will  need  to  be  addressed  by  data.  Some  additional 
data  collection  and  effort  may  be  needed  for  an  expanded  database  use.  Questions 
decision-makers  may  raise  that  would  require  additional  data  include  these:  What 
are  the  year-to-year  trends  for  teaching  particular  subjects  or  courses?  How  many 
service  members  are  in  training  on  a  given  day?  Which  courses  are  most  instruc¬ 
tor-intensive?  How  many  instructors  have  been  used  in  the  past,  and  how  many 
instructor  days  are  planned  for  the  current  year?  How  many  courses  are  currently 
taught  by  contractors?  How  effective  is  distributed  learning?  How  much  school- 
house  teaching  always  will  be  necessary  and  why? 

The  major  elements  of  information  that  senior  managers  need  to  know  in  planning 
distributed  training  are  the  cornerstone  of  the  database.  A  significant  amount  of 
information  has  been  compiled  in  this  database,  enabling  it  to  answer  specific 
training-related  questions:  how  much  goes  on  where?  with  whom?  to  what  end? 
With  additional  budget-related  data,  the  database  could  be  a  powerful  informa¬ 
tional  tool  for  P&R,  answering  real-time  budget  questions. 

We  recommend  that  additional  information  be  collected  that  training  managers 
and  staff  planners  need  to  know  in  transitioning  to  a  distributed  learning  curricu¬ 
lum.  Trainers  need  to  know  what  distributed  learning  courses  and  courseware  are 
available  now,  or  are  being  put  together  coUaboratively,  and  who  future  collabo¬ 
rators  will  be.  In  order  to  invest  wisely  in  distributed  learning,  all  planners  need  to 
know  the  relative  effectiveness  and  costs  of  distributed  learning  curricula  in  com¬ 
parison  to  traditional  methods.  To  develop  a  military-wide  distributed  learning 
plan,  the  existing  course  database  must  be  enhanced  with  information  about  re¬ 
source  use  to  more  accurately  target  the  greatest  areas  of  need. 
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Chapter  1 

Distributed  Learning  Training  Potential 


Introduction 

Advanced  distributed  learning  is  an  exciting  development  in  the  field  of  training, 
with  the  potential  to  improve  the  amount  learned  from  training,  shorten  training 
time,  and  provide  on-the-job  just-in-time  training  for  highly  technical  fields.  The 
capacity  for  sharing  courseware  and  educating  the  workforce  is  enormous  and  is 
enabled  in  large  part  by  electronic  developments  of  the  past  few  years. 

Distributed  learning  is  the  process  of  disseminating  learning  or  instruction  to  nu¬ 
merous  locations.  The  distribution  can  be  synchronous  (at  the  same  time)  or 
asynchronous.  One  example  of  synchronous  distributed  learning  is  videotelecon¬ 
ferencing,  where  one  instructor  is  seen  in  a  number  of  classrooms.  One  subset  of 
distributed  learning  is  distance  learning,  which  usually  implies  distributed  learn¬ 
ing  over  a  large  geographic  distance.  Distance  learning  can  also  be  synchronous 
or  asynchronous.  An  example  of  asynchronous  learning  is  the  recording  of  in¬ 
struction  material  on  a  CD-ROM,  which  is  viewed  by  students  some  time  after  the 
instructor  recorded  the  material. 

This  project  began  as  an  analysis  and  databasing  of  institutional  training  classes  to 
determine  those  courses  ripe  for  consolidation  or  conversion  by  comparing  with 
selected  Army  Distance  Learning  Plan  courses.  This  comparison  process  will  be 
explained  further  in  the  analysis  section.  The  domain  for  comparison  has  ex¬ 
panded  to  comparisons  among  all  the  services’  training  courses,  otiier  agencies’ 
training  courses,  and  commercial  courses.  Additionally,  the  comparison  has 
grown  from  matching  similar  courses  to  matching  parts  of  courses  in  order  to  re¬ 
use  expensive  technological  media. 

This  report  discusses  the  application  of  distributed  learning  technologies  to  insti¬ 
tutional  military  training.  The  paper  proceeds  in  three  chapters.  The  first  chapter 
discusses  distributed  learning  technology.  Chapter  2  describes  the  military  train¬ 
ing  database  that  was  developed  to  begin  to  prioritize  the  transition  from  institu¬ 
tional  training  to  distributed  learning  and  how  it  could  be  expanded  and  further 
used  in  planning  a  transition  for  all  military  services.  It  should  be  understood  that 
the  database  is  complete  as  far  as  the  purposes  for  which  it  was  compiled,  but  that 
it  is  like  a  house  that  begs  expansion,  and  it  could  use  “home  improvements”  in 
some  completed  areas  as  well.  Chapter  3  discusses  the  need  for  return  on  invest¬ 
ment  (ROI)  information  from  distance  learning  conversion,  and  why  ROI  infor¬ 
mation  is  difficult  to  quantify  and  apply.  Chapter  4  makes  reconmiendations:  the 
accomplishment  of  a  self-subscribing  distributed  learning  courseware  network  on 
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the  Internet,  a  study  on  ROI,  and  analyses  of  the  effectiveness  of  distributed 
learning  training  for  the  students. 

The  need  for  assessing  implementation  costs  and  training  effectiveness  gained  is 
foreseen;  so  a  study  plan  is  proposed  to  begin  to  gather  payback  and  training  re¬ 
tention  data  to  be  included  on  a  recommended  electronic  forum. 

What  Are  Advanced  Learning  Technologies? 

Training  managers  throughout  the  public  and  private  sectors  are  implementing  a 
wide  variety  of  advanced  learning  technologies,  ranging  from  multimillion  dollar 
simulations  to  computer  and  video  teletraining.  Virtual  reality,  computer-based 
instruction,  CD-ROM  instruction,  Web-based  correspondence  courses,  electronic 
submittal  of  homework  and  electronic  grading,  automated  self-testing  quizzes, 
and  infinitely  repeatable  computer  simulation  sessions  are  among  the  new  tech¬ 
nologies  available  for  enhancing  training  and  learning.  These  technologies  are  the 
next  generation  of  multimedia  use  in  instruction,  a  logical  extension  of  training 
films,  slides,  and  audio  recordings;  but  also  create  a  new  capability  of  more  real¬ 
istic  instruction  through  simulation.  These  technologies  can  be  inserted  into  ex¬ 
isting  lecture  lessons  or  used  to  supplant  them  entirely. 

Perhaps  the  most  innovative  of  new  technologies  are  distributed  learning  tech¬ 
nologies.  Distance  learning  is  the  accompUshment  of  training  or  education  when 
the  instructor  and  student  are  geographically  separated;  often  in  distributed  learn¬ 
ing  they  are  also  asynchronous,  in  that  the  teacher  creates  instruction  and  the  stu¬ 
dent  receives  instruction  at  separate  times. 

Why  Is  Distributed  Learning  Important? 

Distributed  learning  creates  cost  savings  by  eliminating  or  redueing  the  facilities 
and  travel  eosts  of  geographic  collocation  and  raising  the  student-to-instructor  ra¬ 
tio  without  degrading  instructional  effectiveness.  Distributed  learning  shows 
great  promise  in  maintaining  or  improving  proficiency  and  reducing  cost,  al¬ 
though  these  qualities  are  difficult  to  measure,  and  a  definitive  measuring  of  sav¬ 
ings  for  the  military  has  yet  to  be  made. 

Distributed  learning  technologies  also  hold  the  promise  of  “just-in-time”  training, 
important  particularly  in  technical  fields  where  the  volume  of  information  to  be 
absorbed  eannot  be  obtained  without  extended  and  recurring  training.  The  alter¬ 
native  to  these  time-intensive  training  needs  is  training  on  just  the  topic  needed,  at 
the  time  the  topic  is  needed.  For  example,  an  aircraft  mechanic  who  needs  to  test 
for  fatigue  on  a  particular  part  on  a  specific  aircraft  can  access  the  lesson  on  that 
procedure  on  the  day  he  or  she  needs  the  knowledge,  rather  than  trying  to  learn  all 
possible  repair  and  test  procedures  for  all  aircraft  and  then  refresh  that  knowledge 
annually.  Just-in-time  training  transforms  technical  knowledge  into  on-the-job 
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training,  which  reduces  the  amount  of  time  technicians  need  to  spend  in  training 
and  increases  the  amount  of  time  spent  on  the  job. 

Where  Is  DoD  on  the  Development  Curve? 

One  of  the  largest  training  organizations  in  the  country,  the  U.S.  military  services, 
offer  only  isolated  distributed  learning  courses.  Only  one  service,  the  Army,  has  a 
formal  plan  to  transition  to  distributed  learning  in  training.  The  Army  currently 
has  several  initiatives  underway,  including  the  following:  the  Army  Distance 
Learning  Plan,  which  lists  525  courses  to  convert;  installation  of  electronic  class¬ 
rooms  at  bases  across  the  nation;  and  improvements  in  the  Defense  Information 
System  Network  (DISN)  to  handle  distributed  learning  data  traffic.  Trainers  at 
Fort  Knox  and  possibly  other  installations  have  converted  individual  courses  ac¬ 
cording  to  their  own  needs,  not  as  part  of  the  Army  plan. 

The  Marine  Corps  plans  a  telecommunications  upgrade  and  plans  course-by¬ 
course  conversion  at  the  Marine  Corps  Institute.  The  Air  Force  is  in  a  multiyear 
process  to  develop  a  distributed  learning  program.  The  Air  Force  currently  uses 
interactive  videoteletraining  at  Maxwell,  Shepherd,  and  McGhee  Tyson  Air  Force 
Bases,  using  the  Army  data  backbone;  and  CD-ROM  development  is  going  on  at 
Shepherd,  Lacklund,  Maxwell,  and  Gunter  Air  Force  Bases. 

The  Navy  has  a  program  objective  memoranda  outhning  a  positive  approach  to 
distributed  learning,  has  developed  several  courses,  and  is  using  distributed 
learning  shipboard  through  Shipboard  Training  Education  Advancement  Morale 
(STEAM).  STEAM  is  a  multi-purpose  distribution  system  that  includes  Programs 
for  Afloat  College  Education  (PACE),  Shipboard  Training  Enhancement  Program 
(STEP),  and  several  other  programs  including  morale-builders.  The  Navy  an¬ 
nounced  plans  to  convert  six  courses  a  year  in  the  area  of  developmental  skill 
treiining  and  has  developed  and  offers  a  eourse  for  autiioring  distributed  learning 
called  the  Navy  Interactive  Courseware  Novice  Authoring  Course  (NICNAC). 

The  Navy  Reserves  have  a  common  backbone  in  place,  running  Lotus  Notes  and 
called  “R-Net.”  The  Navy  plans  to  share  the  Marine  common  hardware  suite  and 
Army  bases’  distributed  learning  classrooms. 

In  short,  the  services  are  just  getting  going  on  the  development  curve.  The  Navy 
and  Army  have  2  or  more  years  of  development  behind  them,  but  the  Air  Force 
and  Marines  are  just  now  getting  started.  There  is  a  lot  going  on  outside  DoD  for 
the  services  to  share  or  borrow. 

Distributed  learning  already  has  begun  to  revolutionize  both  education  and  train¬ 
ing.  The  Federal  Acquisition  Institute  and  the  Defense  Acquisition  University 
offer  nearly  entire  curriculums  through  distributed  learning.  Several  other  univer¬ 
sities  around  the  country  have  begun  to  use  distributed  learning,  including  voca¬ 
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tional  schools,  undergraduate  institutions,  and  graduate  schools.^  Large  corpora¬ 
tions  including  Boeing,  Xerox,  and  Lockheed  Martin  were  early  implementers  of 
just-in-time  and  on-demand  training.  In  time,  it  can  be  predicted  that  the  applica¬ 
tion  of  training  technologies  will  broaden  the  audience  and  lower  the  cost  of  edu¬ 
cation  and  training.  Training  and  education  will  be  more  accessible  to  more 
people  throughout  their  lives.  Just  as  the  Internet  has  transformed  libraries  from 
book  depositories  to  linked  information  centers  serving  a  broad  variety  of  media, 
advanced  training  technologies  will  transform  schools  and  universities  into  access 
ramps  for  lifelong  learning  across  unlimited  fields  of  knowledge. 

However,  distributed  learning  is  still  the  exception  rather  than  the  mode  or  even  a 
large  percentage  of  offered  training.  Many  university  colleges  have  no  distributed 
learning  courses.  The  majority  of  people  who  receive  advanced  education  and 
training  do  so  early  in  their  career  from  schoolhouse-based  instruction.  Many  in¬ 
structors  fear  distributed  learning  for  its  potential  for  misapplication  and  lack  of 
instructor  involvement. 

Presently,  each  of  the  above-mentioned  practitioners  offers  its  expertise  to  a  lim¬ 
ited  market;  there  is  no  national  clearinghouse  of  distributed  learning  information. 
Information  on  existing  distributed  learning  or  training  technologies  often  is  found 
through  Internet  searches  or  attendance  at  professional  conferences.  The  search 
for  an  already  developed  training  course  by  existing  practitioners  could  take 
months.  Encouragement  of  the  intervening  steps  between  where  we  are  now  and 
the  vision  of  just-in-time  training  for  everyone  is  discussed  in  this  report.  We  be¬ 
gin  with  the  potential  for  converting  some  of  the  30,000-plus  institutional  courses 
offered  by  the  armed  services.  Some  have  obvious  cost  savings  potential  due  to 
avoided  travel.  Many  courses  could  be  shared  with  corporate  America  and  other 
civilian  government  agencies. 

How  Can  Distributed  Learning  Technologies 
Be  Applied  for  the  Services? 

Most  military  recruits  receive  an  exhaustive  education  in  performance  of  a  par¬ 
ticular  vocation  shortly  after  boot  camp.  The  training  is  given  at  Initial  Skill 
Training;  then,  the  recruit  joins  his/her  unit,  where  a  tiny  percentage  of  the  knowl¬ 
edge  just  learned  is  put  to  use.^  The  recruit  may  have  received  training  sufficient 
to  become,  say,  a  certified  auto  mechanic,  but  on  arrival  at  his/her  unit,  he  or  she 
is  initially  given  less  demanding  tasks,  such  as  routine  maintenance.  At  some 
point  in  the  future,  the  recruit  needs  to  move  on  to  the  next  skill  level,  and  at  that 

’  Distributed  learning  is  known  to  be  offered  in  undergraduate  degree  programs  at  National 
Louis  University  and  Florida  State  University.  Graduate  degree  programs  incorporating  distrib¬ 
uted  learning  include  Virginia  Tech,  George  Mason  University,  Johns  Hopkins  University,  Vir¬ 
ginia  Commonwealth  University,  University  of  Maryland,  and  many  others. 

^  Please  see  Appendix  B  for  a  brief  description  of  the  stages  of  military  training. 
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point  often  is  sent  baek  to  a  training  facility  for  refresher  training,  incurring  mov¬ 
ing  costs,  per  diem,  and  salary.  The  unit  must  try  to  do  without  the  member  while 
training  proceeds.  For  the  Navy,  the  number  of  sailors  that  can  be  working  on  a 
ship  is  limited  by  the  number  stationed  at  training  between  tours  at  sea. 

If  service  members  were  given  Skill  Progression  Training  (SPT)  at  their  unit 
rather  than  returning  to  a  training  center,  moving,  travel,  and  per  diem  costs  could 
all  be  saved,  and  more  services  members  would  be  “ready”  with  their  units  at  any 
given  time.  The  question  now  becomes,  how  can  appropriate  training  be  deliv¬ 
ered  to  the  service  member  at  his  or  her  unit  at  the  proper  time? 

The  Navy  has  begun  to  address  this  training  requirement  through  its  STEAM  pro¬ 
gram,  in  which  it  distributes  training  CD-ROMs  to  ships  once  a  year.  If  all  serv¬ 
ices  were  to  incorporate  some  type  of  SPT  distributed  learning,  a  substantial 
proportion  of  the  20  percent  of  service  members  continually  in  training  could  re¬ 
main  with  their  units  and  their  families  more. 

Training  courseware  can  be  developed  by  the  military,  by  commercial  vendors,  or 
by  corporate  training  centers  and  civilian  government  agencies  with  similar  inter¬ 
ests.  In  fact,  this  development  by  all  these  participants  is  going  on.  For  example, 
a  series  of  public  affairs  officer  courses  developed  with  electronic  media  and 
available  by  correspondence  has  been  developed  by  an  office  in  the  U.S.  Army 
and  Training  Doctrine  Command  (TRADOC),  with  potential  application  to  Navy, 
Marine,  and  Air  Force  public  affairs  officers.  A  CD-ROM  course  on  security  in¬ 
vestigations  developed  by  the  FBI  could  be  used  by  security  officers  in  all  serv¬ 
ices,  if  made  available  to  them.  A  video  course  on  blood  safety  and  blood-bome 
pathogens  developed  commercially  could  be  incorporated  in  institutional  military 
health  care  training. 

Advanced  training  media  and  electronic  files  may  have  been  developed  at  great 
eost  to  assist  in  a  training  course.  They  could  be  used  elsewhere  for  similar 
training  if  the  applicable  training  authority  knew  of  the  existence  of  the  media,  if 
training  planners  could  evaluate  its  value  as  a  training  aid,  and  if  military  training 
managers  had  the  authority  to  access  the  media  and  make  it  part  of  a  course  cur- 
rieulum.  The  challenge  lies  in  discovering  and  organizing  needs,  identifying 
technology  alternatives,  and  making  the  technology  choices  available  to  the  serv¬ 
ices  and  service  members  needing  those  tools. 

In  Chapter  2,  we  discuss  an  effort  that  was  already  begun  in  order  to  determine 
convergent  training  needs  and  to  identify  distributed  learning  opportunities.  The 
data  collection  and  compilation  effort  described  in  Chapter  2  was  a  needed  step  to 
discover  basic  training  priorities,  and  its  accomplishment  contributed  to  the  for¬ 
mation  of  the  Army  Distance  Learning  Plan  eourse  conversion  list.  At  the  end  of 
Chapter  2  and  beginning  of  Chapter  3,  we  discuss  the  next  steps  that  need  to  be 
taken  to  further  encourage  distributed  learning  and  the  steps  to  be  taken  to  identify 
where  distributed  learning  is  most  warranted. 
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Chapter  2 

Building  a  Training  Course  Database 


Initial  Collection  of  Data 

The  database  that  is  discussed  in  this  section  should  be  considered  complete,  in 
that  it  can  answer  the  questions  it  was  compiled  to  answer.  However,  the  database 
is  like  a  house  that  has  room  for  expansion:  other  closely  connected  arrays  of  in¬ 
formation  could  be  added  that  would  allow  the  database  to  answer  many  more 
questions.  This  section  provides  an  accounting  of  the  capabilities  of  the  database 
much  as  an  architectural  drawing  would  show  the  size  and  shape  of  a  house’s 
rooms.  This  description  is  useful  to  anyone  considering  building  or  expanding  a 
database  to  answer  military  training  questions.  The  database  is  also  very  large  in 
relation  to  this  report  and  cannot  reasonably  be  appended.  The  complete  answer 
to  a  single  inquiry  may  run  to  150  pages.  Excerpts  are  provided  throughout  this 
section  to  illustrate  the  contents  of  some  of  the  20  related  files  that  comprise  the 
database. 

LMI  participated  in  a  joint  project  with  for  the  Office  of  Personnel  and  Readiness, 
RAND,  and  the  Institute  for  Defense  Analysis  to  collect  schoolhouse  training 
course  data  and  produce  a  recommended  “short  list”  of  courses  for  consolidation 
with  other  institutional  and  distributed  learning  courses.  The  result  of  the  joint 
project  was  incorporated  into  the  annual  Military  Occupational  and  Training  Data 
report,  in  its  first  index.  ^ 

Information  was  collected  on  current  timing  capabilities  and  characteristics. 
Course  numbers,  course  titles,  owning  service,  services  taking  the  course,  length 
of  course,  the  military  occupational  usefulness  of  completing  the  course,  the  level 
of  training  offered  in  the  course,  consolidation  of  courses,  contract  or  outsourced 
instruction,  and  use  of  technological  courseware  data  was  collected.  We  obtained 
files  from  each  of  the  services  describing  the  institutional  training  courses  offered 
over  the  previous  3  years.  Each  service  has  different  training  priorities.  The  data 
available  in  each  category  varies  because  service-specific  databases  contain  vary¬ 
ing  types  of  information. 

Because  the  data  fields  were  to  be  analyzed  for  grouping  by  subject  matter,  all  oc¬ 
cupational  subject  matter  information  available  was  requested.  At  least  two  files 
from  each  service  were  tendered.  A  single  course  might  be  listed  in  one,  two,  or 


'  Defense  Manpower  Data  Center,  Arlington,  VA,  1995  Military  Occupational  and  Training 
Data,  1995. 
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all  of  a  services’  data  files.  In  the  case  of  the  Marine  Corps,  many  of  the  courses 
listed  are  offered  by  other  services  but  are  open  to  Marine  Corps  enrollment.  Ap¬ 
pendix  A  offers  an  extended  discussion  of  the  contents  of  the  data  files. 

From  the  collected  data,  we  constructed  a  database  encompassing  the  services’ 
and  DoD’s  institutional  skill  training  courses  over  30,000  courses.  Initially,  we 
wanted  to  identify  institutional  training  courses  for  conversion  to  distance  learning 
conversion  for  consolidation.  The  database  also  was  immediately  useful  for  gen¬ 
erating  comparative  and  descriptive  statistics  about  the  vast  field  of  military 
training. 

Using  the  Database  to  Answer  Training- 
Oriented  Questions 

Having  developed  “clusters”  of  similar  courses  across  the  services,  the  most 
promising  clusters  were  evaluated  with  assistance  from  the  Interservice  Training 
and  Requirements  Organization  (TTRO)  so  that  recommendations  for  involving 
contract  instruction  and  technology  insertion  could  be  made.  (See  Figure  2-1  for 
the  health  and  environmental  safety  cluster.)  Military  occupations  were  ranked  by 
the  number  of  courses  they  used  or  required.  Different  training  levels,  called 
Military  Manpower  Training  Requirements  (MMTRs)  were  evaluated  by  LMI  to 
determine  where  training  efficiencies  were  most  needed  and  could  be  applied. 

(The  answer  was  the  area  of  SPT.)  As  new  issues  and  priorities  arose,  the  data 
were  queried  to  determine  the  number  of  flight  courses,  the  number  of  progressive 
skill  training  courses,  and  similar  questions  of  fact  surrounding  the  armed  forces 
training  status. 

One  of  the  questions  asked  was  where  commonalties  with  private  sector  training 
needs  would  be  found:  the  answer  would  provide  P&R  with  starting  points  for 
military-industry  training  sharing.  The  database  was  developed  further  and 
crosswalks  were  developed  to  translate  military  occupations  into  civilian  profes¬ 
sions.  The  various  military  occupational  identifiers  had  been  translated  by  tables 
from  the  Defense  Manpower  Data  Center  (DMDC);  these  were  supplemented 
with  the  Department  of  Labor’s  Standard  Occupational  Codes.  Furthermore,  oc¬ 
cupational  crosswalking  was  supplemented  by  subject  content  categorization  on 
the  logic  that  dissimilar  occupations  may  share  common  skill  sets  and  desire  to 
share  course  materials. 
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Figure  2-L  Example  of  an  Occupational  Cluster 


ENVIRONMENTAL  HEALTH  AND  SAFETY  SPECIALISTS 

MOTDOCC  1473011  THROUGHPUT 

MANPOWER  4,500  ACCESSIONS  350 

ALTERNATE  TITLES 

ENVIRONMENTAL  HEALTH  SPECIALISTS;  ENVIRONMENTAL  SCIENTISTS;  AIR  POLLUTION 
ANALYSTS;  SOILS  ANALYSTS;  INDUSTRIAL  HYGIENISTS;  AIR  QUALITY  ANALYSTS;  FOOD  AND 
DRUG  INSPECTORS;  PUBLIC  HEALTH  INSPECTORS. _ 

GENERAL  DESCRIPTION 

Each  military  base  is  a  small  community.  The  health  and  well-being  of  the  residents  and  surrounding  land 
is  a  major  concern  of  the  services.  Keeping  military  work  places  and  living  areas  sanitary  helps  to  prevent 
illness.  Environmental  health  and  safety  specialists  inspect  military  facilities  and  food  supplies  for  the  pres¬ 
ence  of  disease,  germs,  or  other  conditions  hazardous  to  health  and  the  environment. 

WHAT  THEY  DO 

Environmental  health  and  safety  specialists  in  the  military  perform  some  of  all  of  the  following  duties: 
Monitor  storage,  transportation,  and  disposal  of  hazardous  waste; 

Analyze  food  and  water  samples  to  ensure  quality; 

Conduct  health  and  safety  investigations  of  living  quarters  and  base  facilities; 

Provide  training  on  industrial  hygiene,  environmental  health,  and  occupational  health  issues; 

Monitor  noise  and  radiation  levels  at  job  sites. 

TRAINING  PROVIDED 

Job  training  consists  of  1 1  to  19  weeks  of  classroom  instruction,  including  practice  in  making  health  and 
sanitation  inspections.  Training  length  varies  depending  on  specialty.  Course  content  typically  includes 
Identification  of  health  hazards; 

Inspection  of  food  products  and  food  sen/ice  operations; 

Inspection  of  wastewater  and  waste  disposal  facilities; 

Further  training  occurs  on  the  job  and  through  advanced  courses. 

CIVILIAN  COUNTERPARTS 

Most  civilian  environmental  health  and  safety  specialists  work  for  local,  state,  and  federal  government 
agencies.  Their  duties  are  similar  to  the  duties  of  military  environmental  health  specialists.  They  may  be 
called  food  and  drug  Inspectors,  public  health  inspectors,  health  and  safety  inspectors,  or  industrial  hy¬ 
gienists. 

Air  Force 

4B0X1  BIOENVIRONMENTAL  ENGINEERING  JOURNEYMAN 

3E4X3  ENVIRONMENTAL  JOURNEYMAN 

4E0X1  PUBLIC  HEALTH  JOURNEYMAN 

Navy 

9595  HAZARDOUS  MATERIAL  CONTROL  MANAGEMENT  TECHNICIAN 

8432  PREVENTIVE  MEDICINE  TECHNICIAN 

9591  RADIOLOGICAL  CONTROL  MONITOR 

Army 

91 S  PREVENTIVE  MEDICINE  SPECIALIST 

91 R  VETERINARY  FOOD  INSPECTION  SPECIALIST 

Marines 

9954  HAZARDOUS  MATERIAUHAZARDOUS  WASTE  SPECIALIST 

8033  QUALITY  ASSURANCE  TECHNCIAN  (SUBSISTENCE) 
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Interest  in  resource  sharing  between  the  military  and  commercial  companies  be¬ 
came  an  item  for  collaboration.  The  database  was  again  utilized  to  help  discover 
broad  areas  of  commonality.  Occupational  matching  was  grouped  and  added  to 
the  database.  (See  Appendix  C  for  occupational  systems  descriptions.)  Addition¬ 
ally,  because  dissimilar  occupations  might  have  some  convergent  training  inter¬ 
ests,  a  course  content  matching  capability  was  designed  and  implemented. 

LMI  investigated  many  existing  methods  for  content  categorization,  including  the 
following:  the  Dewey  decimal  system;  university  course  numbering  systems;  the 
Library  of  Congress  numbering  system;  the  Department  of  Commerce’s  Standard 
Industrial  Codes;  the  Office  of  Personnel  Management’s  Knowledge,  Skills,  and 
Abilities;  the  United  States  Air  Force  Occupational  Measurement  Squadron’s 
(AFOMS)  Occupational  Analysis  Program;  DoD  contractor  contributions  to 
AFOMS;  the  Department  of  Labor’s  Occupational  Information  Networks 
(0*NET)  content  matrix;  and  several  custom-designed  educational  media  catego¬ 
rizations  published  by  the  Department  of  Education.^ 

Requirements  for  a  content  categorization  were  derived  on  the  basis  of  the  func¬ 
tional  needs  of  the  database  and  the  expected  analyses.  The  categorization  had  to 
be  numerically  based  or  translatable  to  a  number  system  to  use  in  database  format. 
The  categorization  had  to  be  hierarchical  rather  than  dictionary-based  in  order  to 
find  near  matches  as  well  as  exact  matches  and  allow  the  construction  of  broad 
groups  for  comparison  and  possible  consolidation  of  courses  by  subject  matter. 
The  categorization  had  to  encompass  a  broad  subject  matter:  vocational  training 
for  the  military,  professional  and  university  training  for  the  growing  service  com¬ 
ponent  of  the  military  and  the  economy,  and  remedial  (e.g.,  adult  reading)  and 
child  education,  two  useful  areas  for  the  military.  With  the  increasing  demand  to 
reuse  electronic  learning  media,  the  categorization  had  to  be  applicable  to  whole 
study  areas  (e.g.,  degrees),  courses,  lessons,  and  learning  aids  (e.g.,  video  clips). 
Finally,  we  wanted  to  use  an  existing  system  if  possible  to  save  time  and  improve 
its  acceptance  into  use. 

The  International  Standard  Classification  of  Education  (ISCED),  devised  by  the 
United  Nations  Educational,  Scientific,  and  Cultural  Organization,  seemed  to  best 
satisfy  the  requirements.  The  ISCED  structure  is  described  in  detail  in  Appendix 

^  0*NET  was  at  this  point  in  time  a  placeholder.  No  draft  categorization  was  available. 

^  Including,  but  not  limited  to  the  following:  Far  West  Laboratory  for  Educational  Research 
and  Development,  Berkeley,  CA,  Rationale  and  Schedule  for  a  Classification  System  for  Educa¬ 
tion,  U.S.  Department  of  Health,  Education,  and  Welfare,  1972;  Kellogg  Foundation,  Kellogg 
Adult  Education  Thesaurus^  Battle  Creek,  MI.,  1997;  Office  of  Career  Education,  Policy  Studies  in 
Education,  Taxonomy  and  Profiles  of  Career  Education,  New  York,  NY :  Department  of  Health, 
Education  and  Welfare,  1997;  United  Nations  Educational,  Scientific,  and  Cultural  Organization, 
International  Standard  Classification  of  Education  (ISCED),  New  York,  NY:  United  Nations, 
1973. 
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D,  as  well  as  the  recommended  ways  to  apply  ISCED  to  course  matching  and 
electronic  media  reuse.  Course  content  information  is  useful  in  recording  and 
comparing  lessons  and  distributed  learning  materials;  it  also  promotes  the  substi¬ 
tution  of  technology  for  classroom  time  and  encourages  sharing  of  electronic  re¬ 
sources,  which  will  save  money  for  developers. 
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Chapter  3 

Projected  Analyses  of  the  Database:  Capabilities 


Information  Available  Now 

The  above  narrative  should  give  the  reader  a  pretty  good  indication  of  the  separate 
fields  available  in  the  database,  and  thus  guide  questions  that  could  be  asked  of  it. 
The  database  already  can  provide  some  information  to  help  decide  where  priori¬ 
ties  lie.  The  database  can  describe  the  course  subjects  that  are  most  frequently 
offered  in  schoolhouse  training,  in  toto  or  by  service.  We  can  describe  how  geo¬ 
graphically  dispersed  the  training  is  by  showing  how  many  courses  are  offered,  at 
which  schools,  and  where  those  schools  are  located.  (See  Figure  3-1.)  Using  load 
information,  we  can  describe  roughly  whether  enlisted  persons,  officers,  or  other 
people  comprise  the  majority  of  students  by  course  subject  matter.  The  database 
can  reveal  which  courses  are  most  often  taken  by  pilots,  by  medical  students,  and 
by  other  career  choices —  and  how  long  the  coursework  takes.  In  the  current  state 
of  high  utilization  of  certain  military  specialties  such  as  pilots,  this  database  could 
begin  to  answer  ways  in  which  to  reduce  pilot  training  and  thus  increase  the  avail¬ 
ability  of  pilots  for  duty.  As  budget  planners  examine  base  realignment  and  clos¬ 
ings,  this  database  can  help  illustrate  where  alternative  schoolhouse  training  is 
available  and  where  it  could  be  lacking. 

Information  the  Database  Can  Provide  as  It 
Matures 

With  some  additional  data  collection  and  effort,  there  are  some  likely  policy  and 
budget  questions  the  database  could  answer.  Some  information  exists  in  the  data¬ 
base  showing  training  load  figures  from  FY95  to  FY97;  but  in  other  areas,  only  1 
year  of  information  might  have  been  provided.  If  training  start  dates  and  year-to- 
year  loads  were  included,  the  database  could  be  used  to  project  the  number  of 
service  members  in  training  at  any  one  time,  how  the  load  changes  through  the 
course  of  the  year,  and  when  the  active  force  is  most  drained  by  training. 

In  preparing  distributed  learning  courses,  training  planners  should  be  concerned 
with  the  year-to-year  trends  for  teaching  particular  subjects  or  courses,  in  order  to 
know  which  courses  are  rising  in  demand  and  are  good  candidates  for  conversion. 
The  same  number  of  instructors  could  be  used  to  teach  the  increasing  number  of 
students  expected. 
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Figure  3-1.  Locations  of  Military  Training  Schools  by  Service 


If  a  distributed  learning  courseware  database  can  be  created  with  links  to  the 
schooUiouse  training  database,  new  opportunities  for  finding  cost  savings  instantly 
present  themselves.  New  courseware  developers  can  be  sent  a  list  of  the  schools 
offering  courses  similar  to  one  they  may  be  developing.  An  analysis  of  the 
courses  that  are  most  instructor-intensive  and  those  that  do  not  convert  well  to 
distributed  learning  can  be  conducted.  Tracking  how  distributed  learning  is  im¬ 
plemented  gives  us  a  basis  with  which  to  estimate  future  personnel  and  budgetary 
requirements. 

Existing  distributed  learning  practitioners  have  noted  that  in  the  first  year,  distrib¬ 
uted  learning  does  not  always  pay  for  itself,  but  it  has  obvious  readiness  advan¬ 
tages  that  win  the  day  for  implementation.  Return  on  investment  for  analysis 
needs  to  be  conducted  for  distributed  learning  practitioners  to  answer  questions 
like  these:  How  many  days  are  being  reduced  from  the  training  schedule  each 
year?  How  do  distributed  learning  students  test  against  traditional  schoolhouse 
students?  What  is  the  best  instructor-to-student  ratio  in  distributed  learning,  and 
which  topics  need  more  attention  than  others?  How  long  does  it  take  for  distrib¬ 
uted  learning  to  pay  off?  What  payoffs  can  be  expected  in  different  fields?  Does 
the  retention  rate  in  distributed  learning  courseware  fall  off  after  a  period  of  time, 
and  what  does  this  imply  for  courseware  refreshment  rates? 

It  should  be  emphasized  that  return  on  investment  is  far  from  a  simple  matter 
when  talking  about  course  consolidation  or  conversion.  In  the  institutional  data¬ 
base,  a  simple  query  reveals  a  number  of  courses  that  all  offer  instruction  in  envi- 
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ronmental  health  and  safety  (see  Table  3-1.)  Many  courses  share  the  same  course 
number  or  description.  However,  some  are  offered  in  geographically  disparate 
locations  and  consolidating  them  by  bringing  students  together  would  be  more 
costly  than  holding  1 1  different  classes.  If  a  change  is  made  in  instruction  that 
allows  shorter  classes,  but  the  same  number  of  instructors  and  same  capital  facili¬ 
ties  are  used,  are  there  any  savings?  If  an  instructor  develops  computer-based  in¬ 
struction  and  100  more  people  take  the  course  each  year  than  before,  how  should 
that  be  expressed  monetarily?  Many  courses  are  also  offered  several  times  each 
year  and  are  part  of  a  series  that  make  up  a  curriculum.  If  one  course  is  consoli¬ 
dated  or  converted  so  that  it  can  be  taught  once  annually,  there  are  arguably  no 
savings  since  students  must  wait  for  an  open  slot  in  the  next  course  in  the  series. 

The  major  elements  of  information  that  senior  managers  need  to  know  in  planning 
distributed  training  are  the  cornerstone  of  the  database.  A  significant  amount  of 
information  has  been  compiled  in  this  database,  enabling  it  to  answer  several  spe¬ 
cific  training-related  questions:  how  much  training  goes  on  and  where?  with 
whom?  to  what  end?  With  additional  budget-related  data,  the  database  could  be  a 
powerful  informational  tool  for  P&R,  answering  real-time  budget  questions. 
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Table  3-1.  Courses  in  Environmental  Health  and  Safety 


Course  ID 

Title 

MOTD 

Code 

MOC/ 

NEC 

Air  Force 

X3ABR9S200 

APPLIED  SCIENCES 

1473011 

9S2X0 

Air  Force 

J3ABR3E433 

ENVIRONMENTAL  APR 

1473011 

3E4X3 

Air  Force 

L3AQR9S200 

ELECTRONIC  PRINCIPLES  (EP) 

1473011 

9S2X0 

Air  Force 

J4ART3E453 

PEST  MGMT  SPLST  (RECERT) 

1473011 

Air  Force 

J5AZA4B071 

RADIOLOGICAL  SAFETY  (4J-F3/494-F14) 

1473011 

Air  Force 

J3AZR3E453 

ENVIRONMENTAL  MNGMT  (CERTIFICATION) 

1473011 

Air  Force 

J1ASC3E453 

ENVIRONMENTAUPEST  MANAGEMENT  TRANSITION 

1473011 

Air  Force 

J3ARR3E453 

MANAGEMENT  JOURNEYMAN 

1473011 

Air  Force 

J2AST3E453 

MOBIL  ENVIRONMENTAL  TRANSITION 

1473011 

Air  Force 

J5AZA4E071 

SEXUALLY  TRANS  &  OTHER  COMM  DISEASE 

1473011 

INTERVENTION 

Air  Force 

J3AZR3E453 

ENVIROMENTAL  TRANSITION 

1473011 

Air  Force 

J3AZR3E453 

ENVIRONMENTAL  MNGMT  (CERTIFICATION) 

1473011 

Air  Force 

J3ABR3E433 

ENVIRONMENTAL  APR 

1473011 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

N 

avy 

A-322-2600 

HMC&M  TECH 

9595 

Army 

321-91R10 

Veterinary  Food  Inspection  Specialist  (Basic) 

1473011 

Army 

321-91R10 

Veterinary  Food  Inspection  Specialist  (Basic)  (RC)(PH  2) 

1473011 

Army 

322-91 S10 

Preventive  Medicine  Specialist 

1473011 

Army 

322-91  SI  0 

Preventive  Medicine  Specialist  (RC)(PH  2) 

1473011 

Army 

6-8-C40(91R30) 

AMEDD  NCO  Basic  (NCOES) 

1473011 

Army 

6-8-C40(91S30) 

AMEDD  NCO  Basic  (NCOES) 

1473011 

Army 

321 -91  RIO 

Veterinary  Food  Inspection  Specialist  (Basic) 

91R1 

Army 

321 -91  RIO 

Veterinary  Food  Inspection  Specialist  (Basic)  (RC) 

91R1 

Army 

6-8-C40(91R30) 

AMEDD  NCO  Basic  (NCOES) 

91 R3 

Army 

322-91  SI  0 

Preventive  Medicine  Specialist 

91S1 

Army 

322-91  SI  0 

Preventive  Medicine  Specialist  (RC) 

91S1 

Army 

6-8-C40(91S30) 

AMEDD  NCO  Basic  (NCOES) 

91  S3 

Chapter  4 

Visions  for  the  Future 


Distributed  Learning  Development  Questions 
Needing  Answers 

Training  managers  today  face  the  following  issues: 

♦  What  are  the  opportunities  for  distributed  learning,  both  today  and  in  the 
near  future? 

♦  Which  training  programs  can  benefit  the  most  from  the  use  of  distributed 
learning?  Does  that  benefit  involve  cost  reduction  or  more  effective 
learning? 

♦  What  incentives  are  there  to  develop  distributed  learning? 

♦  How  much  is  the  government  spending  on  distributed  learning  technolo¬ 
gies?  Is  this  investment  paying  off? 

This  report  discusses  the  work  undertaken  to  begin  to  answer  these  questions. 
This  paper  outlines  the  structure  of  an  information  and  analytical  capability  for 
training  managers  facing  the  tough  choices  of  whether  and  where  to  invest  in  dis¬ 
tributed  learning  training  technologies.  The  development  of  a  prototype  informa¬ 
tion  and  analysis  system  is  a  key  recommendation  for  subsequent  research. 

Other  information  that  future  research  should  address  includes  these  questions: 

♦  What  kind  of  data  do  training  managers  need  to  make  investment  deci¬ 
sions? 

♦  What  data  are  available  to  answer  their  questions? 

♦  What  are  the  risks  and  how  are  they  mitigated? 

♦  How  can  we  distribute  the  information  DoD  already  has? 
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After  training  managers  have  identified  the  range  of  opportunities  available  to 
them,  they  then  face  the  task  of  choosing  the  best  technologies  and  options  for 
their  needs.  Managers  are  expected  to  understand  their  training  needs  but  may 
desire  assistance  in  addressing  investments  in  distance  learning.  For  example,  a 
typical  new  developer  must  answer  these  questions: 

♦  Who  will  develop  the  courseware?  Should  we  outsource? 

♦  What  media  should  we  use?  How  much  will  it  cost? 

♦  What  hardware  facilities  are  available  to  support  the  technological 
coxu'seware?  Will  we  have  to  buy  new  equipment?  How  much  will  it  cost 
over  its  lifetime? 

♦  How  will  courseware  be  distributed  or  delivered? 

♦  When  should  courseware  be  updated? 

♦  What  is  the  degree  of  instruction  intervention  with  this  courseware?  Does 
it  make  the  instructor’s  job  easier  or  harder? 

♦  How  do  we  measure  if  the  technology  is  effective?  Will  the  students  learn 
the  materizd? 

♦  How  much  will  we  really  save? 

♦  If  we  reach  more  students,  what  next? 

These  issues  should  be  addressed  in  future  research,  which  is  laid  out  more  fully 
in  the  section  entitled  Next  Steps. 

Assessing  Technology  Insertion  Payback 

How  much  of  an  improvement  can  be  expected  from  technology  infusion  over 
traditional  methods?  What  is  the  true  payback  of  technology  use  in  training? 
Technological  media  is  expensive  to  create,  and  accounting  payback  analyses 
show  that  few  course  conversions  will  pay  for  themselves.  These  analyses  fail  to 
capture  all  of  the  benefits  of  course  conversion  due  to  the  lack  of  measurement  in 
improvement  of  human  capital. 

Cost  savings  studies  show  that  technology  insertion  is  accompanied  by  a  decrease 
in  instruction  hours.*  Concurrently,  a  single  instructor  has  greater  availability  to  a 
larger  number  of  students.  Typically,  the  distributed  learning  instructor  can 


*  See  Evaluation  of  Pilot  Distance  Learning  Course,  Operational  Training  Unit,  National  Se¬ 
curity  Division,  Federal  Bureau  of  Investigation,  June  1997. 
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instruct  four  times  as  many  students  as  an  instructor  who  can  use  only  school- 
house  instruction.  Traditional  payback  analyses  measure  the  decrease  in  class 
time,  the  savings  in  student  and  instructor  salaries  or  value  of  time,  travel  time, 
administrative  costs,  and  processing  costs.  These  figures  are  all  fairly  concrete 
and  agreeable  measurements.  But  experience  also  shows  that  technology  insertion 
in  the  classroom  increases  student  participation,  absorption  level,  and  retention 
rates.  Students  learn  the  material  better,  score  better  on  comparative  tests,  and 
retain  the  material  longer.  The  improvements  observed  in  technology-assisted 
courses  are  similar  to  the  results  seen  with  individual  tutoring.  We  will  attempt  to 
quantify  the  training  effectiveness  improvements  implied  in  this  better  result;  it  is 
the  student’s  payback. 

In  studying  training  effectiveness,  we  also  will  be  developing  payback  analyses 
that  can  be  used  not  only  for  evaluating  the  implementation  of  distributed  learning 
technologies  but  also  for  ascribing  cost-sharing  arrangements  when  two  agencies, 
or  an  agency  and  a  private  company,  agree  to  develop  a  course  together  for  mutual 
use. 

Recommendations  for  Continued  Course 
Technology  Integration 

Growth  of  the  Database 

This  task  initially  began  as  an  analysis  and  databasing  of  institutional  training 
classes  to  determine  those  courses  ripe  for  consolidation  or  comparison  with 
Army  distance  learning  courses  with  close  comparisons.  The  population  for  com¬ 
parison  has  expanded  beyond  the  Army  to  all  the  services,  other  agencies,  and 
commercial  courses.  Additionally,  the  comparison  has  grown  from  matching 
similar  courses  to  matching  lessons  and  parts  of  lessons,  especially  those  involv¬ 
ing  the  use  of  expensive  technological  media.  Sharing  learning  media  requires 
identifying  and  cataloging  course  media. 

Distance  Learning  Data 

The  course  database  should  be  expanded  to  include  existing  DoD  distributed 
learning  courseware,  to  capture  both  lessons  learned  and  opportunities  for  media 
reuse.  A  great  number  of  DoD  service  members  already  have  prepared  distributed 
learning  courses  and  media,  and  the  group  grows  continuously.  The  growth  of 
technology  courseware  is  so  fast  in  fact  that  other  distributed  learning  practitio¬ 
ners  are  unaware  of  resources  that  may  be  available  for  sharing  and  reuse.  Tradi¬ 
tional  database  and  publishing  methods  for  publicizing  courseware  will  always  lag 
well  behind  the  actual  course  availability. 


Courseware  Information  Resources  Now 


The  Defense  Instructional  Technology  Information  System  (DITIS)  maintains  a 
formal  database  of  distributed  learning  information  and  has  information  standards 
and  an  established  republishing  cycle.  Formal  standards  for  media  and  course 
categorization,  storage,  and  sharing  are  being  discussed  in  the  Advanced  Distrib¬ 
uted  Learning  Forum.  The  formal  standardization  process  is  necessary  but  delays 
by  months  or  years  the  dissemination  of  course  information  that  may  have  a  shelf 
life  of  only  3  years. 

Additionally,  many  service  members  and  distributed  learning  developers  have 
mentioned  that  they  wished  they  had  access  to  information  about  the  scope  of 
training  in  DoD.  When  writing  a  course  for  their  unit  (for  example,  a  repair  re¬ 
fresher  course),  it  would  be  useful  to  know  how  many  service  members  complete 
basic  repair  courses  each  year  and  might  want  to  use  the  refresher  in  the  future,  or 
that  another  unit  already  had  developed  such  a  tool. 

To  redress  the  problem  of  immediately  being  able  to  locate  distributed  learning 
reuse  opportunities  in  DoD,  a  self-subscribing  Web  page  should  be  established.  A 
database  loaded  onto  the  World  Wide  Web  can  be  automatically  updated  with  an 
electronic  submittal  form  sent  by  e-mail.  People  with  access  to  the  Internet  would 
merely  need  to  fill  out  a  template  with  service  affiliation,  course  number,  length, 
types  of  instmction,  course  content  or  keywords,  price  of  access  (if  any),  and 
point-of-contact  information  and  e-mail  it  to  an  automatic  server.  Text- 
recognition  facsimile  submission  provides  an  effective  substitute  for  those  who 
are  “Internet  deprived”,  and  any  corrections  can  be  pointed  out  via  e-mail  for  a 
human  operator  to  incorporate. 

This  modest  proposal  would  provide  a  hare-bones  forum  for  finding  available 
distributed  learning  and  technology-enhanced  instruction.  When  all  the  issues  for 
content  classification  have  been  ironed  out,  a  basic  list  of  courseware  will  be 
waiting  for  application. 

Next  Steps 

A  summary  of  the  requirements  set  out  above  must  encompass,  first  and  foremost, 
the  building  of  an  interactive  information  tool.  An  interactive  Web  page  must  be 
launched;  simultaneously  a  distributed  learning  database  will  begin  to  be  estab¬ 
lished. 

The  Advanced  Distributed  Learning  Forum  will  work  with  DoD  and  industry  par¬ 
ticipants  to  flesh  out  content  definitions — ^those  given  here  are  not  definitive  and 
do  not  represent  an  accepted  standard;  they  are  only  the  product  of  a  research  pro¬ 
gram. 
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Key  Elements  of  a  Distributed  Learning 
Information  and  Analysis  System 

The  proposed  interactive  page  and  DoD  distributed  learning  database  provide  a 
good  foothold  for  bringing  together  practitioners  to  answer  the  questions  outlined 
in  the  Distributed  Learning  Development  Questions  Needing  Answers  section 
above.  A  natural  extension  of  the  utility  and  the  next  step  that  developers  will 
need  will  be  the  development  and  analysis  of  information  about  courses  that  will 
help  managers  make  good  business  decisions. 

LMI  plans  to  begin  collecting  data  on  Web  course  instruction  costs,  visit 
“corporate  universities,”  and  begin  researching  distributed  learning  effectiveness 
to  develop  information  databases  and  simple  models  for  managers  to  access.  At 
its  most  basic,  the  research  will  provide  answers  to  “how  much  will  it  cost?”  and 
“how  much  will  my  department  save?”  But  such  research  is  also  fundamental  to 
solving  the  following  problems: 

♦  If  tutoring  improves  average  learning  performance  by  two  standard  devia¬ 
tions,  how  much  does  distributed  learning  improve  average  performance? 
What  causes  the  difference?  Is  it  robust  over  time? 

♦  H  my  department  and  another  civilian  agency  develop  a  course  together, 
how  do  we  decide  how  much  each  should  pay  in  acquisition  and  life-cycle 
costs? 

In  summary,  an  electronic  forum  would  offer  the  distributed  learning  community  a 
place  to  share  courseware  and  media,  leam  about  implementation,  discover  the 
financial  information  needed  to  launch  a  distributed  learning  project,  and  research 
the  methods  for  creating  electronic  media.  It  may  also  provide  the  beginnings  of 
collaboration  for  DoD  and  civilian,  private,  and  corporate  training  managers,  and 
scholarly  work  on  how  training  is  effective. 
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Appendix  A 

Training  Database 


The  database  currently  consists  of  14  files.  For  each  service,  there  is  an  index  of 
course  numbers  with  Military  Manpower  Training  Requirement(MMTRs)  and, 
often,  the  course  title.  Each  service  has  several  files  with  varying  information,  but 
each  generally  contains  at  a  minimum,  course  identification  numbers,  course  title, 
MMTR  code  (indicates  level  of  training),  the  service  “owning”  the  course,  and 
services  taking  the  course.  In  many  cases,  data  also  are  available  on  the  number 
of  students  enrolled  in  the  course;  when  and  where  (school  code)  the  course  was 
offered;  whether  the  course  was  offered  through  a  contractor,  was  outsourced,  or 
used  computer-based  instruction;  how  many  students  enrolled  and  graduated  the 
course  in  each  of  the  past  2  years;  the  occupational  specialty  represented;  and 
course  content.  Additionally,  several  files  that  translate  occupational  codes  from 
Military  Occupational  Codes  (MOCs)  to  Military  Occupational  and  Training  Data 
(MOTD)  to  Standard  Occupational  Classification  (SOC)  are  considered  part  of  the 
accumulated  database  capability. 

Often,  the  same  course  is  listed  more  than  once;  if  offered  in  a  different  location 
or  to  a  different  audience,  sometimes  the  course  is  renamed  and  given  a  new 
course  number.  Some  duplicate  “different  location  but  same  course”  records  exist 
in  LMI’s  database  of  schoolhouse  training,  but  they  are  relatively  easy  to  sort  out 
if  needed.  Each  service  has  its  own  identification  number  for  its  courses,  even 
tiiose  offered  by  another  service;  so,  if  a  Marine  recruit  takes  an  Army  course, 
there  will  likely  be  two  listings  of  the  same  course  with  two  different  course  iden¬ 
tification  numbers.  This  double-counting  across  services  as  yet  has  not  been 
remedied,  though  comparison  of  course  title  and  content  would  turn  up  several  of 
them.  Including  the  duplicates  noted  above,  approximately  18,332  courses  are  in 
the  database.  (See  Table  A-1.) 
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Table  A-1.  Breakout  of  Military  Courses  by  Skill  Training  Category 


Number  Of  Unique  Courses 

Service 

Total 

Recruit,  OSUT, 
officer  training 

1ST,  SPT,  and 
flight  training 

University 

education 

Army 

5,551 

819 

4,543 

184 

Air  Force 

6,657 

0 

6,566 

35 

Marine  Corps 

1,562 

0 

1,231 

97 

Navy 

2,527 

10 

2,440 

3 

It  should  be  noted  that  at  this  point,  the  data  in  this  database  is  2  years  old  and 
may  be  incomplete.  It  cannot  be  assumed  that  it  encompasses  all  military  school- 
house  training  courses,  though  every  method  to  include  all  possible  courses  has 
been  made,  given  the  time  and  materials  available. 

The  database  includes  variables  likely  to  support  analytical  requirements  in  budget 
and  policy  decision-making,  such  as  load,  training  level,  service  owning  the 
course,  services  participating  in  the  course,  technology  insertion,  contracting 
status  and  co-location.  Training  data  in  this  database  was  collected  initially  by 
each  service  in  support  of  staffing  planning  requirements. 

Fifty-five  percent  of  courses  listed  in  the  military  schoolhouse  class  database  have 
some  sort  of  occupational  identifier,  ranging  fi'om  a  high  of  99.7  percent  for  Army 
courses  to  a  low  of  7  percent  for  Marine  Corps  courses. 
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Appendix  B 

Overview  of  Traditional  Military  Training 


Military  training  follows  a  general  sequence  that  is  common  across  all  services. 
Upon  call-up,  recruits,  or  enlisted  personnel,  are  given  a  physical  and  sent  to  re¬ 
cruit  training.  Often  called  “boot  camp,”  recruit  training  indoctrinates  enlistees  in 
the  procedures  and  regimen  of  military  life.  Recruits  receive  combat  training,  fit¬ 
ness  training,  and  begin  to  receive  service-specific  information.  Recruit  training 
is  conducted  by  each  service  in  specified  locations;  for  example,  the  Marine  Corps 
trains  recruits  only  at  Parris  Island  and  San  Diego.  (See  Table  B-1.)  Following 
recruit  training,  about  half  of  Navy  recruits  and  a  small  percentage  of  other  serv¬ 
ices’  recruits  are  sent  to  their  units;  the  rest  begin  specialized  skill  training,  begin¬ 
ning  with  Initial  Skill  Training  (1ST).  1ST  consists  of  apprentice-level 
occupational  training  and  can  last  from  3  to  52  weeks,  but  averages  7.5  weeks. 

The  Army  offers  1ST  at  the  same  site  as  recruit  training  in  about  half  their  training 
facilities,  a  practice  called  One  Station  Unit  Training  (OSUT).  After  1ST,  most 
recruits  then  go  to  their  units,  though  some  receive  additional  training. 

Table  B-1.  Recruit/Basic  Training 


Service 

School  location 

Weeks  of 
training 

Air  Force 

Lackland  AFB,  TX 

6 

Marine  Corps 

San  Diego,  CA  (men  only) 

11 

Marine  Corps 

Parris  Island,  SC 

12 

Navy 

Great  Lakes  School,  IL 

9 

Army 

Combat  only 

Ft.  Knox,  KY 

8 

Army 

Combat  only 

Ft.  Sill,  OK 

8 

Army 

Combat  only 

Ft.  Leonard  Wood,  MO 

8 

Army 

Combat  only 

Ft.  Jackson,  SC 

8 

Army 

Combat  +  1ST  (OSUT) 

Ft.  Knox,  KY 

11  + 

Army 

Combat  +  1ST  (OSUT) 

Ft.  Sill,  OK 

11  + 

Army 

Combat  +  1ST  (OSUT) 

Ft.  Leonard  Wood,  MO 

11  + 

Army 

Combat  +  1ST  (OSUT) 

Ft.  Banning,  GA 

11  + 

Army 

Combat  +  1ST  (OSUT) 

Ft.  McClellan,  AL 

11  + 
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After  a  period  of  time  with  their  units,  depending  on  unit  occupational  staffing 
needs  and  term  of  service,  some  percentage  of  enlisted  personnel  are  sent  on 
travel  or  temporary  tour  of  duties  to  training  facilities  for  SPT. 

SPT  courses  may  hold  the  highest  return  on  investment  for  conversion  to  distrib¬ 
uted  learning,  since  SPT  usually  involves  traveling  from  the  enlistee’s  regular  unit 
to  a  distant  location.  Pilots  and  flight  officers  receive  a  separate  regimen  of 
training,  consisting  of  undergraduate  flight  training,  flight  familiarization  training, 
advanced  flight  training,  and  other  flight  training.  Professional  development  edu¬ 
cation  is  also  tracked  by  DoD.  DoD  subsidizes  education  at  professional  military 
schools  and  universities,  predominantly  legal  education,  health  professions  edu¬ 
cation,  undergraduate  education,  and  graduate  education.  Officers  in  all  services 
receive  precommissioning  training,  also  called  Officer  Acquisition  Training 
(OAT). 

Most  military-funded  educational  (university)  courses  are  omitted  from  military 
training  course  rosters  for  a  variety  of  reasons.  Military  service  members  attend 
civilian  universities,  so  the  spectrum  of  available  courses  is  too  broad  to  be  worth 
recording  in  a  database;  and  these  courses  are  not  owned  by  any  service  or  DoD. 
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Samples  of  Occupational  Crosswalks 


Occupational  Codes 

Most  military  services  categorize  their  courses  and  positions  by  occupational 
groups.  The  exception  is  the  Marine  Corps;  most  of  its  courses  currently  are  not 
mapped  to  occupational  groups.  Several  categorization  schemes  are  used;  the 
major  ones  are  Military  Occupational  and  Training  Data  (MOTDs)  Occupational 
Employment  Statistics/Dictionary  of  Titles  (OES/DOTs),  and  Standard  Occupa¬ 
tional  Classification  (SOCs)  and  Military  Occupational  Codes  (MOCs).  Each 
classification  scheme  is  described  briefly  below. 

The  MOTDs  categorization,  organized  by  the  Defense  Manpower  Data  Center 
(DMDC),  consists  of  7-digit  numerical  codes,  called  MOTDs  or  MOTDOCCs 
(MOTD  Occupational  Codes).  A  total  of  152  MOTD  specialties  apply  to  military 
personnel,  grouped  into  1 1  broad  categories.  Generally,  the  last  digit  of  the  7- 
digit  code  indicates  whether  the  occupation  is  filled  by  enlisted  personnel  or  offi¬ 
cers. 

The  OES/DOT  was  a  classification  system  used  by  the  Department  of  Labor 
(DOL).  The  185  OES  codes  were  five  numerical  digits  in  length,  and  did  not 
match  well  with  MOTDs.  Several  MOTDs  could  fall  under  one  OES  while  some 
MOTDs  fell  under  multiple  OESs.  DOT/OES  is  no  longer  used  by  DOL;  it  has 
been  replaced  with  the  SOC. 

The  SOC  formerly  consisted  of  two  to  four  digits  depending  on  the  level  of  detail 
of  classification  desired;  in  1997  a  major  revision  was  begun  and  all  codes  will  be 
about  six  digits  in  length.  In  the  1997  SOC  there  are  22  “major  groups”  of  occu¬ 
pations,  around  100  “minor  groups”  that  correspond  well  with  MOTDs,  then  hun¬ 
dreds  of  “broad  occupations”  and  even  more  “detailed  occupations.”  An 
illustration  of  the  1997  SOC  follows: 

17-0000  ARCHITECTURE  AND  ENGINEERING  OCCUPATIONS  (Major 
Group) 

17-2000  ENGINEERS  (Minor  Group) 

17-2010  ELECTRICAL  AND  ELECTRONICS  ENGINEERS  (Broad 
Occupation) 

17-2011  ELECTRICAL  ENGINEERS  (Detailed  Occupation) 
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MOCs  are  widely  used  and  difficult  to  explain.^  The  MOCs  have  anywhere  from 
two  to  nine  digits,  numbers  and  characters,  and  they  represent  very  generalized  to 
very  specialized  occupational  titles.  MOCs  are  amalgamated  from  every  service’s 
own  occupational  codes,  including  the  Army’s  Military  Occupational  Specialty 
(MOS),  Air  Force  Specialty  Codes  (AFSCs),  Navy  Officer  Billet  Classifications 
(NOBCs),  and  Navy  Enlisted  Classifications(NECs).  Within  a  service’s  occupa¬ 
tional  classification,  there  are  subcategories  such  as  Area  of  Concentration 
(AOC),  Additional  Skill  Identifier  (ASI),  and  Special  Qualification  Identifier 
(SQI);  each  corresponds  to  a  smaller  number  of  digits  within  the  larger  code. 

Each  service  uses  a  different  set  of  MOCs,  sometimes  using  different  codes  for 
the  same  title.  MOCs  are  meant  to  be  fairly  detailed  descriptions  and  often  20  or 
more  MOCs  will  correspond  to  one  MOTD.  For  example,  an  Army  Administra¬ 
tive  Assistant  (MOC  71L),  a  Marine  Personnel  Clerk  (0121),  a  Marine  Legal 
Services  Specialist  (4421),  and  a  Navy  Legalman  Second  Class  (LN),  all  fit  under 
the  MOTD  4630991  for  Administrative  Support  Specialists. 

Military  Occupational  Specialty  (MOS)  codes  were  established  by  the  Army,  and 
after  some  period  of  use.,  the  services  adopted  modified  versions  of  the  Army’s 
MOS.  Some  modifications  were  extreme,  and  the  codes  continue  to  shift  over 
time.  Air  Force  Specialty  Codes  (AFSC)  were  developed  in  1951,  revised  in 
1993,  and  correspond  to  only  nine  career  groups.  AFSCs  have  a  different  number 
of  digits  for  officers  and  enlisted  personnel.  The  Navy  has  two  separate  code 
groups,  the  NOBCs  and  NECs,  for  officers  and  enlistees. 


'  The  Defense  Manpower  Data  Center  (DMDC)  tracks  these  codes  and  regularly  updates 
them.  For  translation  services,  contact  the  DMDC.  John  Fowlkes  of  the  DMDC-East  has  prepared 
a  concise  and  understandable  six-page  summary  of  the  derivation  of  MOC  codes,  an  excellent  ref¬ 
erence. 
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Appendix  D 

Content  Definitions 


Content  Classification 

Content  classification  may  be  used  in  two  ways:  to  compare  plans  of  instruction 
and  to  sort  and  identify  technological  learning  media.  A  general  scheme  was  re¬ 
ferred  to  in  the  main  text  that  will  be  expanded  upon  below.  Full  use  of  the  Inter¬ 
national  Standard  Classification  of  Education  (ISCED)  scheme  will  likely  involve 
the  addition  of  a  keyword  system  to  create  fuller  levels  of  detail.  A  keyword  sys¬ 
tem  seems  inescapable  in  detailed  media  classification  due  to  the  possible  content 
variety.  Numbers  are  not  recommended  since  they  are  nonmnemonic,  and  the 
general  grouping  that  numbering  would  accomplish  has  already  been  addressed  at 
the  ISCED  level. 

To  illustrate  the  possible  use  of  ISCED  and  keyword  classification,  consider  the 
example  of  an  instructor  writing  a  course  in  operations  research.  The  instructor 
wishes  to  access  electronic  media  items  related  to  “queuing  theory.”  The  instruc¬ 
tor  will  first  narrow  his  search  to  the  ISCED  category  of  “math  and  computer  sci¬ 
ence,”  and  then  search  for  otiier  courses  or  for  electronic  media  under  specific 
headings,  such  as  “queuing  theory.”  If  the  courseware  preparer  is  unable  to  think 
of  appropriate  keywords,  he  or  she  should  be  able  to  look  up  queuing  theory  in  the 
synonym  dictionary  and  find  the  appropriate  keywords,  which  may  be  “delay 
models.” 

Keywords  are  also  used  to  prevent  classified  items  from  falling  into  oblivion.  The 
first  impulse  of  contributors  to  the  reusable  electronic  media  database  may  be  to 
use  words  such  as  “highway  traffic  congestion  simulation,”  “modified  Monte 
Carlo  analysis,”  “C++  program,”  and  “client-server  optimization  strategies,”  so 
media  should  be  classed  in  established  topic  areas,  like  “delay  model,”  to  increase 
the  items’  visibility.  As  classification  is  taken  to  finer  and  finer  gradations,  the 
possibility  of  reuse  is  greater,  but  at  the  price  of  a  concomitant  increase  in  admin¬ 
istrative  effort.  For  example,  a  portfolio  of  medical  illustrations  may  be  classed  as 
“various  medical  illustrations,”  but  the  media  will  be  more  accessible  if  more  spe¬ 
cific  descriptors  are  applied,  such  as  “illustrations  of  muscle  groups”  and 
“illustrations  of  joints  of  the  body.” 

These  classification  recommendations  are  the  product  of  a  research  effort  and  are 
proposed  to  encourage  the  development  of  classification  systems  in  the  distributed 
learning  field.  While  the  ISCED  is  fairly  complete,  the  keyword  dictionary  has 
been  left  undeveloped  and  general  recommendations  are  made  about  its  construc¬ 
tion,  since  the  idea  of  using  a  keyword  dictionary  has  yet  to  be  widely  reviewed. 
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Other  Media  Categorization  Needs 


Guidelines  are  needed  in  electronic  media  to  encourage  reuse  of  categorized 
items.  Courseware  developers  will  not  try  to  reuse  categorized  media  if  the  items 
are  unusable  due  to  inappropriate  size  or  platform  constraints.  A  key  issue  at  the 
time  of  this  writing  is,  what  is  the  size  of  an  electronic  media  item?  Is  a  media 
item  a  video  clip,  a  scanned  photo  or  diagram,  or  a  3-hour  interactive  computer 
course?  If  our  goal  is  to  encourage  deposits  and  reuse  in  the  media  marketplace, 
then  the  answer  must  be:  a  size  that  is  convenient  to  the  developer  and  to  the  user. 
Furthermore,  standards  for  electronic  files  must  be  established;  requiring  use  of 
the  hypertext  markup  language  (i.e.,  HTML)  language  is  an  example.  These  is¬ 
sues  are  raised  only  as  issues  recommended  for  resolution  when  categorizing 
items  by  content. 

The  U.S.  Department  of  Education  recommends  classifying  educational  aids  by 
the  following  six  categories.^  Subcategorical  descriptors  are  drawn  from  the  De¬ 
partment  of  Education  or  from  ISCED,  as  noted. 

1 .  Title  or  brief  description  of  material 

2.  Source  and  date  of  material 

3.  Educational  level;  the  following  two  alternative  systems  are  available: 


DoE 

ISCED 

Preschool 

0  =  preceding  first  level  (ages  3-5) 

Early  elementary  (ages  5-7) 

1  =  education  at  the  first  level  (ages  5-1 1 ;  also  reme¬ 
dial  learning) 

Upper  elementary  (ages  8-12) 

2  =  education  at  the  second  level,  first  stage  (ages 
11-13) 

Junior/middle  (ages  12-14) 

3  =  education  at  the  second  level,  second  stage 
(ages  14-16) 

Senior  (ages  15-17) 

5  =  education  at  the  third  level,  first  stage,  non¬ 
university  degree  (ages  17+;  associates  degrees  and 
trade  training) 

Post  secondary 

6  =  education  at  the  third  level,  first  stage,  university 
degree 

Adult 

7  =  education  at  the  third  level,  second  stage;  post 
graduate  work 

9  =  education  not  definable  by  level:  seminars,  rec¬ 
reation,  professional  skills 

'  Office  of  Career  Education,  Department  of  Health,  Education  and  Welfare,  A  Classification 
Scheme  for  Career  Education  Resource  Materials,  Washington,  D.C.:  GPO,  June  1975,  p.  14. 

Categories  are  drawn  from  “A  Classification  Scheme...”  but  subcategories  have  been  edited 
for  social  and  technological  change. 
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Content  Definitions 


4.  Discipline  and  content  area  (see  ISCED  schedule,  below) 

5.  Special  audience 

♦  Specific  language 

♦  Braille 

♦  Large  type 

♦  etc. 

6.  Type  of  material 

♦  Tools  and  related  items 

♦  Games 

♦  Puzzles 

♦  Activity  kits 

♦  Videotapes 

♦  TransparenciesA^u-graphs 

♦  Slides 

♦  Records/audio  tapes 

♦  Films  (various  sizes) 

♦  Textbook 

♦  Teacher’s  guide 

♦  Student  workbook 

♦  Simulation,  computer-based 

♦  Simulation,  desk-based 

♦  Laboratory  practice 

♦  Evaluation  practice  (quizzes  and  exams) 

♦  Bibliography 

♦  Directory 

♦  Look-up  tables 

♦  Administrative  material 

♦  Photos 

♦  Audio  CD 

♦  Film  CD 

♦  Text  CD/CD-ROM 

♦  Interactive  CD 

♦  Physical  demonstration  model  (e.g.,  medical  mannequin) 

♦  Electronic  files,  various  formats 

♦  Virtual  reality  device 

♦  etc. 
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Course  Matching 

Courses  need  to  be  matched  to  each  other  by  subject  so  that  distributed  learning 
courses  can  be  accessed  by  trainers  in  the  same  field.  Additionally,  courseware 
preparers  who  are  searching  for  preexisting  media  to  use  must  be  able  to  find  me¬ 
dia  by  subject.  Thus,  for  courseware  classification,  a  multilayer  system  also  is 
proposed,  beginning  with  the  ISCED  subject  fields  then  use  of  a  keyword  struc¬ 
ture  and  dictionary.  That  system  looks  like  this: 

♦  Proposed  layers 

>►  Subject  matter  field  (one  of  20  choices) 

>►  Course  subject  (one  or  more  of  500  choices) 

>-  Course  objectives  by  keywords  (several  words  from  keyword  diction¬ 
ary) 

♦  ISCED  subject  fields 

►  01-10  =  K-1 2  fields 
>-  14  =  teacher  training 

►  18  =  fine  and  applied  arts 
>-  22  =  humanities 

>-  26  =  religion  and  theology 

►  30  =  social  and  behavioral  science 

>-  34  =  commercial  and  business  administration 

►  38  =  law  and  jurisprudence 

►  42  =  natural  science 

►  46  =  math  and  computer  science 
>-  50  =  medical  and  paramedical 

>-  54  =  engineering 

>-  58  =  architecture  and  town  planning 

>-  62  =  agriculture,  forestry,  fishery 
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. . _  Content  Definitions 

>■  66  =  home  economics  and  domestic  science 

►  70  =  transport  and  communication 

►  14  =  trade,  craft,  and  industrial 

>■  80  =  programs  for  the  handicapped 

►  99  =  other 

►  48  =  space 

►  76  =  weapons 

Application  of  ISCED  to  Military  Training 
Data 


The  training  database  contains  limited  course  content  information  from  the  Army 
Training  Requirements  and  Resources  System  (ATRRS).  Course  content  infor¬ 
mation  is  useful  in  recording  and  comparing  lessons  and  distributed  learning  ma¬ 
terials;  it  promotes  the  substitution  of  technology  for  classroom  time  and 
encourages  sharing  of  electronic  resources.  It  is  anticipated  that  additional  content 
information,  if  needed,  could  be  collected  from  the  Army  through  ATRRS-Total 
Army  Centralized  Individual  Training  Solicitation  (TACITS);  from  the  Navy 
(Pensacola);  Air  Force  (Randolph  AFB);  and  from  the  Marine  Corps  (Quantico). 

In  the  existing  database,  content  matching  may  enable  whole  courses  to  be  shared. 
Even  without  an  occupational  definition,  content  matching  enables  identification 
of  courses  with  common  training  objectives,  such  as  wheeled  vehicle  maintenance 
in  the  Army  and  vehicle  maintenance  training  by  General  Motors  Corporation. 

Once  a  market  for  electronic  media  is  established,  the  content  schema  can  be  used 
to  identify  electronic  media  for  reuse.  For  example,  a  computer-generated  graphic 
of  storm  cloud  development  may  get  reuse  in  related  meteorology  courses  and  in 
flight  training.  The  electronic  media  “marketplace”  may  be  established  as  a 
linked  part  of  the  DoD  training  database  or  in  a  new  incarnation,  perhaps  in  a  dis¬ 
tributed  netware  Internet  site. 
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Appendix  E 

DoD  Distributed  Learning  Points  of  Contact 


This  appendix  lists  points  of  contact  in  DoD  for  already-developed  distributed 
learning  courses.  It  is  a  preliminary  list.  Following  the  DoD  distance  learning 
course  contact  list  are  DoD  distributed  learning  universities,  points  of  contact  for 
already-developed  civilian  government  courses,  DoD  distributed  learning  web 
link  sites,  and  potential  somces  of  technological  media. 

DoD  Distributed  Learning  Courses 

♦  Intranet-based  course  on  Army  warfare  strategy 

Luciano  lorizzo,  Jr. 

Technical  Advisor,  USAARMC 

ATSB-DL,  Bldg  2369,  Fort  Knox,  KY,  40121-5200 

iorizzol  @  ftknox-dtdd-emh5.army.mil 

♦  Internet  courses 

Offered  free.  Subjects:  Acquisition  Reform,  Safety,  Education,  Tech¬ 
nical  Training,  Communication  Skills,  Organization  Skills,  Manage¬ 
ment,  Career/Personal  Development,  EEO,  Macintosh  Skills 

Marilyn  Newman 
Training  Specialist 

AEGIS  Training  and  Readiness  Center 
5396  First  St. 

Dahlgren,  VA  22448-5200 

Maiilyn_Newman.at.AEGIS-TRACEN@hq.navsea.navy.mil 

♦  CD-ROM  officer/leadership  course 

MSgt  Leslie  E.  Amidon,  BGS  and  MSgt  Frank  A.  Mileto,  B.S. 
Interactive  Courseware  Instructional  Designers 
550  McDonald  St. 

Maxwell  Air  Force  Base 

Gunter  Annex,  AL  361 14-3107 

http://www.au.af.mil/au/cepme/college.htm 

♦  CD-ROMs:  Shipboard  Training  Enhancement  Program  (STEP);  offers  a 
wide  variety  of  topics  including  Global  Positioning  Satellite  System,  Cor- 
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rosion  Control,  Cooking,  Lookout  Training,  Pump  Maintenance.  STEP  is 
one  of  many  such  programs  offered  by  the  Navy. 

www.cnet.navy.mil/netpdtc/step/ 

♦  Teleconferencing  courses 

Public  Affairs  Officer  Training 
Edith  Alexander,  Ph.D. 

American  Forces  Information  Service 

Office  of  the  Assistant  Secretary  of  Defense  (Public  Affairs) 

Directorate  of  Training 

601  N.  Fairfax  Street 

Alexandria,  VA  22314-2007 

eealexa@hq.afis.osd.mil 

♦  A  variety  of  materials  are  available  for  training  and  refreshment  on  acqui¬ 
sition  including  Web  files,  video  tapes,  and  recordings. 

Acquisition  Reform  Resource  Center  learning  materials 

http://afmc.wpafb.af.mil/organizations/HQ- AP^C/DP/dpe/lb- 
9/arrc.html 

♦  Correspondence  courses  on  logistics,  personnel  retention,  supervision, 
mobilization,  alcohol  and  drug  abuse  by  the  Army  Readiness  Reserve 
Training  Center. 

http://host.Id.centuryinter.net/arrtc/decentrl.html 

DoD  Distributed  Learning  Universities 

♦  Air  University 

Courses  are  available  in  Civil  Air  Patrol,  Military  Education,  Officer 
Training,  Air  Strategy,  Aerospace  Doctrine  and  Research,  Air  Force  Judge 
Advocates,  Military  Comptroller,  Chaplain  Service,  First  Sergeant  Acad¬ 
emy,  Historian  Development,  etc.;  offered  via  Web  or  CD-ROM. 

http://www-acsc.au.af.mil 

♦  Defense  Acquisition  University 

Courses  for  acquisition  officers 
http://www.acq.osd.mil/dau/dau.html 
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DoD  Distributed  Learning  Points  of  Contact 


♦  Naval  Postgraduate  School 

M.S.E.E.,  Electrical  and  Computer  Engineering 
Prof.  Gerry  Thierren 

Department  of  Electrical  and  Computer  Engineering 

Naval  Postgraduate  School,  Monterey,  California  93943 

http://vislab-www.nps.navy.mil/~mtummala/distleam/distance.html 

♦  National  Defense  University 

A  number  of  colleges  at  NDU  offer  distributed  learning 
http://www.ndu.edu/homepage/html/ 

Civilian  Government  Agencies’  Distributed 
Learning  Programs 

♦  CD-ROM  Basic  Security  Course,  converted  from  institutional  training. 

Patricia  M.  Boord 

Curriculum  &  Interactive  Training  Technologies  Program  Manager 
National  Security  Training 
FBI  Academy 
Quantico,  VA  22135 

♦  Interactive  Continuing  Medical  Education 

Birmingham  Veterans  Affairs  Medical  Center  is  installing  a  T-1  based 
video  teleconferencing  system  that  will  hnk  primary  care  providers  at 
all  VAMCs  in  the  state  of  Alabama.  Video  teleconferencing  equipment 
will  be  placed  in  the  treatment  areas  of  the  hospitals  for  consults  with 
specialists. 

Tom  Nielsen,  Ed.D. 

Associate  Director 

Birmingham  Regional  Medical  Education  Center 

Medical  Foram  Suite  500 

950  North  22nd  Street 

Birmingham,  AL  35203 

nielsentom  @  lm.va.gov 

Also  see  http://www.va.gov/mediauto/telemed  for  similar  initiatives  in 
all  states. 
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♦  Distributed  Learning  for  Emergency  Medical  Service  Technicians 


Garry  B.  Criddle,  R.N.,  CDR.,  USCG/US  PHS 

Emergency  Medical  Services  Specialist 

Executive  Secretary  MAST 

U.S.  Department  of  Transportation 

National  Highway  Traffic  Safety  Administration 

400  7th  Street,  S.W.  (NTS-42) 

Washington,  D.C.  20590 
Gcriddle@NHTSA.DOT.GOV 

DoD  Distributed  Learning  Web  Sites 

♦  http://www.atsc-army.org/ 

♦  http://www..ngbs..ngb.army.mil/.tng/Arsites.htm#dod 

♦  http://www.ott.navy.niil  (Chief  of  Naval  Operations  Office  of  Training  Tech¬ 
nology,  Seamless  Product  Information  Data  Exchange  and  Repository) 

♦  http://atscweb.atsc-army.org/accp/aipd.htm  (Army  Training  Support  Center) 

♦  www.dcst.monroe.army.mil/adlp/wbt-l.htm  (Status  Report  on  Use  of  the  In¬ 
ternet  and  World  Wide  Web  for  delivery  of  Army  Distributed  Learning,  de¬ 
veloped  by  Training  Development  Analysis  Activity  Headquarters,  U.S.  Army 
Training  and  Doctrine  Command,  Fort  Monroe,  VA) 

DoD  Audio-Visual  and  Broadcasting 
Materials 

♦  http://www.defenselink.mi1/afis/index.html#tasa  (American  Forces  Informa¬ 
tion  Service) 

♦  http://michp753.redstone.army.mil/davis/  (Defense  Automated  Visual  Infor¬ 
mation  System) 

♦  www.dmdc.osd.mil/ditis/  (Defense  Instructional  Technology  Information 
System  (DITIS)  database  to  facilitate  resource  sharing  within  DoD  and  reduce 
costs  associated  with  redundant  interactive  courseware  (ICW)  development.) 

♦  http://xenon.brooks.af.mil/  HSC/AL/SD/SDS.html  (Technical  Visual  Services 
provides  professional  and  artistic  support,  management,  and  products  in  all 
presentation  mediums  for  scientific,  medical,  and  educational  programs  and 
missions  at  the  Armstrong  Laboratory,  Human  Systems  Center,  and  other  DoD 
organizations  such  as  the  Armed  Forces  Institute  of  Pathology.  Mediums  pro- 
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DoD  Distributed  Learning  Points  of  Contact 


duced  include  television,  photography,  illustrations,  video  teleconferencing, 
and  audio  visuals.) 
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Appendix  F 

Abbreviations 


ADLP 

AFB 

ATOMS 

AFSC 

AOC 

ASI 

ATRRS 

ATSC 

CD 

CD-ROM 

DAU 

DISK 

Dins 

DMDC 

DoD 

DoE 

DoL 

DOT 

EEO 

FBI 

Ft 

GPO 

HTML 

ICW 

ISCED 

1ST 


Army  Distance  Learning  Plan 
Air  Force  Base 

United  States  Air  Force  Occupational  Measurement 
Squadron 

Air  Force  Specialty  Codes 
Area  of  Concentration 
Additional  Skill  Identifier 

Army  Training  Requirements  and  Resources  System 
Army  Training  Support  Center 
compact  disk 

compact  disk — ^read  only  memory 

Defense  Acquisition  University 

Defense  Information  System  Network 

Defense  Instructional  Technology  Information  System 

Defense  Manpower  Data  Center 

Department  of  Defense 

Department  of  Education 

Department  of  Labor 

Dictionary  of  Titles 

Equal  Employment  Opportunity 

Federal  Bureau  of  Investigation 

Fort 

General  Printing  Office 
h3q)ertext  mark  up  language 
Interactive  Courseware 

International  Standard  Classification  of  Education 
Initial  Skill  Training 
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ITRO 

KSA 

MMTR 

MOC 

MOS 

MOTD 

MOTDOCC 

NDU 

NEC 

NICNAC 

NOBC 

0*NET 

GAP 

OAT 

OES 

0PM 

OSUT 

P&R 

PACE 


Interservice  Training  and  Requirements  Organization 

Knowledge,  Skills,  and  Abilities 

Military  Manpower  Training  Requirements 

Military  Occupational  Codes 

Military  Occupational  Specialty 

Military  Occupational  and  Training  Data 

Military  Occupational  and  Training  Data  Occupational 
Codes 

National  Defense  University 
Navy  Enlisted  Classifications 

Navy  Interactive  Courseware  Novice  Authoring  Course 

Navy  Officer  Billet  Classifications 

Occupational  Information  Network 

Occupational  Analysis  Program 

Officer  Acquisition  Training 

Occupational  Employment  Statistics 

Office  of  Personnel  Management 

One  Station  Unit  Training 

Personnel  and  Readiness 

Programs  for  Afloat  College  Education 


R-Net  Army  Reserve  Network 


ROI  return  on  investment 

SIC  Standard  Industrial  Codes 

SOC  Standard  Occupational  Codes 

SPIDER  Seamless  Product  Information  Data  Exchange  and  Re¬ 
pository 

SPT  Skill  Progression  Training 

SQI  Special  Qualification  Identifier 

STEAM  Shipboard  Training  Education  Advancement  Morale 

STEP  Shipboard  Training  Enhancement  Program 

TACITS  Total  Army  Centralized  Individual  Training  Solicitation 

TRADOC  U.S.  Army  and  Training  Doctrine  Conunand 
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Abbreviations 


UNESCO 

USAARMC 

USD[P«&R] 

VAMC 


United  Nations  Educational,  Scientific,  and  Cultural  Or¬ 
ganization 

United  States  Army  Armor  Center 

Under  Secretary  of  Defense  for  Personnel  and  Readiness 

Veteran’s  Affairs  Medical  Center 
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REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OPMNo.0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources 
gathering,  and  maintaining  the  data  needed,  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of 
information,  including  suggestions  for  reducing  diis  burden,  to  Washington  Headquarters  Services,  Directorate  for  information  Operations  and  Reports,  1215  Jefferson  Davis  Highway, 
Suite  1204,  Arlington,  VA  22202*4302,  and  to  the  Office  of  Information  and  Regulatory  Affairs,  Office  of  Management  and  Budget,  Washington,  DC  20503, 


1 .  AGENCY  USE  ONLY  (Leave  Blank) 


2.  REPORT  DATE 


3.  REPORT  TYPE  AND  DATES  COVERED 


May  98 


4.  TITLE  AND  SUBTITLE 


5.  FUNDING  NUMBERS 


A  Database  for  Decision-Making  in  Training  and  Distributed  Learning  Technology 


C  DASW01-95-C-0019 
PE  0902198D 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Logistics  Management  Institute 
2000  Corporate  Ridge 
McLean,  VA  22102-7805 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

LMI-PR702T1 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

Office  of  the  Secretary  of  Defense,  Under  Secretary  for  Personnel  and  Readiness 


10.  SPONSORING/MONITORING 
AGENCY  REPORT  NUMBER 


12a.  DISTRIBUTION/AVAILABILITY  STATEMENT 


12b.  DISTRIBUTION  CODE 


A:  Approved  for  public  release;  distribution  unlimited 


13.  ABSTRACT  (Maximum  200  words) 


LMI  created  a  relational  database  to  examine  institutional  training  issues  with  an  emphasis  on  converting  classroom 
training  to  distributed  learning  courses.  The  database  was  used  to  cluster  "schoolhouse"  courses  to  identify  subject 
groups  for  possible  consolidation  and  to  identify  the  courses  promising  the  highest  return  if  converted  to  distributed 
learning.  Distributed  learning  course  conversion  requirements  are  outlined  and  conversion  issues  highlighted.  A 
framework  for  incorporating  data  about  distributed  learning  courseware  into  the  existing  training  database  was 
devised  and  a  plan  for  a  national  electronic  courseware  redistribution  network  was  recommended.  (Electronic 
courseware  is  software  that  enables  distributed  learning.)  The  report  describes  information  now  available  in  the 
institutional  training  database,  potential  benefits  of  distributed  learning  conversion,  and  difficulties  in  assessing 
return  on  investment  from  the  conversion.  Military  training  stages,  courseware  classification  schemes  by  content  and 
occupation,  and  existing  military  distributed  learning  practitioners  are  also  described. 
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